The present disclosure relates in general to artificial neural networks (ANNs) formed from analog signal processing circuitry that models the information processing functionality of a human brain. More specifically, the present disclosure relates to ANNs having current mirror driver circuitry that suppresses voltage change at a charging node of an integrating neuron circuit, thereby maintaining constant current between upstream weighted synapse circuitry and downstream integrating neuron circuitry.
The neuron is a central element of biological neural systems. It has been estimated that the average human brain contains about 100 billion neurons, and, on average, each neuron is connected through fibers to about 1000 other neurons. These neurons and their interconnections form vast and complex biological neural networks that are the mainstay of the brain's processing capabilities. Neurons are remarkable among the cells of the body in their ability to propagate signals such as spike trains or action potentials rapidly over large distances. So-called sensory neurons change their activities by firing sequences of spike trains in various temporal patterns in response to the presence of external stimuli, such as light, sound, taste, smell and touch. Information about a stimulus is encoded in this pattern of action potentials and transmitted into and around the brain.
In biological neural systems, the point of contact between an axon of one neuron and a dendrite on another neuron is called a synapse. With respect to the synapse, the two neurons are respectively called pre-synaptic neuron and post-synaptic neuron. Neurons, when activated by sufficient inputs received via synapses, emit spikes that are delivered to those synapses. Neurons can be either “excitatory” or “inhibitory.” Synaptic conductance is a measure of the influence a synapse will have on its post-synaptic target neuron when the synapse is activated by a pre-synaptic neuron's spike. A person's mental possession of individual experiences is stored in the conductance of synapses which determines the amount of the spike transmission across the trillions of synapses throughout the brain.
The human brain has many desirable characteristics not present in contemporary computer systems, including but not limited to massive parallelism, distributed representation and computation, learning ability, generalization ability, adaptability, inherent contextual information processing, fault tolerance and low energy consumption. Contemporary digital computers outperform humans in the domain of numeric computation and related symbol manipulation. However, humans can effortlessly solve complex perceptual problems (e.g., recognizing an acquaintance in a crowd from a mere glimpse of the person's face) at a speed and to an extent that dwarfs the fastest computer. A reason for such a remarkable difference in the performance of biological neural systems and computers is that the biological neural system architecture is completely different from a typical computer system architecture. This difference significantly affects the types of functions each computational model can best perform.
ANNs are a family of statistical learning models inspired by the biological neural networks of animals, and in particular the brain. ANNs are often embodied as so-called “neuromorphic” systems of interconnected processor elements that act as simulated “neurons” and exchange “messages” between each other in the form of electronic signals. Similar to the so-called “plasticity” of synaptic neurotransmitter connections that carry messages between biological neurons, the connections in ANNs that carry electronic messages between simulated neurons are provided with synaptic weights that correspond to the strength or weakness of a given connection. The weights can be adjusted and tuned based on experience, making ANNs adaptive to inputs and capable of learning. For example, an ANN for handwriting recognition is defined by a set of input neurons that may be activated by the pixels of an input image. After being weighted and transformed by a function determined by the network's designer, the activations of these input neurons are then passed to other downstream neurons, which are often referred to as “hidden” neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character was read.
Crossbar arrays, also known as crosspoint arrays or crosswire arrays, are high density, low cost circuit architectures used to form a variety of electronic circuits and devices, including ANN architectures, neuromorphic microchips and ultra-high density nonvolatile memory. A basic crossbar array configuration includes a set of conductive row wires and a set of conductive column wires formed to intersect the set of conductive row wires. The intersections between the two sets of wires are separated by so-called crosspoint devices, which in effect, function as the synapse components and provide the weighted connections between neuron components.
FIG. 1 depicts a neural network system 100, which is an example configuration of an ANN. System 100 includes a synapse array 102 (e.g., a crossbar array) formed from a plurality of weighted synapse components 104 and neuron components 110, 112, 114, 116, 118, configured and arranged as shown. Neuron components 110, 112, 114, 116, 118, which are shown as Neuronn-1, Neuronn-2, Neuronm, Neuron1 and Neuron0 in FIG. 1, communicate signals (e.g., spike trains) to the plurality of weighted synapse components 104 over axon lines 140 (i.e., conductive column wires) and receives weighted and summed signals over dendrite lines 120 (i.e., conductive row wires). Dendrite lines 120 include dendrite lines 122, 124, 126, 128, 130, which are shown as dendriten-1, dendriten-2, dendritem, dendrite1 and dendrite0 in FIG. 1. Axon lines 140 include axon lines 142, 144, 146, 148, 150, which are shown as axonn-1, axonn-2, axonm, axon1 and axon0 in FIG. 1.
System 100 is a low power system that is implemented using analog signal processing circuitry to model the neuron and synapse components (e.g., 110, 112, 114, 116, 118, 104) of system 100. The analog signal processing is used to model the generation and communication of signals through synaptic connections between neuron components. Output current generated and transmitted through system 100 is used for the control of downstream circuit components. For example, in system 100, synapse current flowing into or flowing out from axon lines 140 provides the stimulus for downstream neuron components. Because system 100 is low power, small unintended variations in certain system parameters can cause errors. For example, system parameters such as the control current on dendrite lines 120 flowing into neuron components 110, 112, 114, 116, 118 are in a critical path and must be independent of other system parameters such as the voltages at the nodes on dendrite lines 120 leading into neuron components 110, 112, 114, 116, 118. Undesired changes in the current on dendrite lines 120 due to the dendrite node voltage may be the result of channel length modulation effects of the MOSFETs which are used as circuit devices to implement neuron and synapse components 110, 112, 114, 116, 118, 104. The channel length modulation effect in MOSFET I-V characteristic is shown in FIG. 5.
It would be beneficial to provide ANN circuitry, wherein key system parameters (e.g., dendrite current into neuron components) are not affected by the undesired effects that can result from undesired variations in such key system parameters.